Bloom-filter fast-path for the dense tag store (O(1) insertion) (phase 2)#11900
Closed
dougqh wants to merge 1 commit into
Closed
Bloom-filter fast-path for the dense tag store (O(1) insertion) (phase 2)#11900dougqh wants to merge 1 commit into
dougqh wants to merge 1 commit into
Conversation
dougqh
added
comp: core
Contributor
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
dougqh
force-pushed
the
dougqh/dense-store
branch
from
July 15, 2026 20:32
27d42d4 to
cd8b73a
Compare
dougqh
force-pushed
the
dougqh/dense-store
branch
from
July 15, 2026 20:41
cd8b73a to
a5e12e4
Compare
dougqh
force-pushed
the
dougqh/dense-bloom-filter
branch
from
July 15, 2026 20:44
428e72f to
267ebad
Compare
|
🎯 Code Coverage (details) 🔗 Commit SHA: 75950be | Docs | Datadog PR Page | Give us feedback! |
dougqh
force-pushed
the
dougqh/dense-store
branch
from
July 15, 2026 21:23
a5e12e4 to
6c70848
Compare
dougqh
force-pushed
the
dougqh/dense-bloom-filter
branch
from
July 15, 2026 21:23
267ebad to
56550cb
Compare
dougqh
force-pushed
the
dougqh/dense-store
branch
from
July 20, 2026 16:49
6c70848 to
0c0d740
Compare
…scan) A per-map long bitmask (knownBloom) over the dense store: a set bit means a tagId MAY be present (scan to confirm), a clear bit means DEFINITELY absent — so the common per-build insert skips the linear knownIndexOf scan and appends in O(1). Crude position->bit map (fieldPos & 63); a collision-minimizing per-type coloring later only raises the hit rate — correctness never depends on it because the scan stays authoritative. Superset semantics: set on add, never cleared on remove (a stale bit costs a scan, never a wrong answer). Alloc-neutral (one long field, no extra allocation); the win is insertion CPU, moving the dense store toward HashMap insertion parity without the scan. Reconciled onto the folded-class dense store (#11814). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
dougqh
force-pushed
the
dougqh/dense-bloom-filter
branch
from
July 20, 2026 16:51
56550cb to
75950be
Compare
dougqh
changed the title
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a per-map bloom-style presence filter (one
long,knownBloom) toOptimizedTagMap's dense known-tag store. A clear bit means a known tag is definitely absent, so an append skips the O(n)knownIndexOflinear scan — the common per-span-build case (each tag set once) becomes O(1).Correctness — superset invariant
The filter is a superset of the present ids' bits: set on every add, never cleared on remove (a stale-set bit only costs an unnecessary scan, never a wrong answer). The linear scan stays authoritative — a set bit means "maybe present, scan to confirm." So correctness is independent of the position→bit collision rate; only the fast-path hit rate depends on it. All 339
TagMap+ dense fuzz tests pass.Position→bit mapping
Crude
1L << (fieldPos & 63)to start.fieldPos(notglobalSerial) is deliberate: it's the axis a future per-type graph-coloring optimizes — coloring co-occurring tags to distinct positions raises the fast-path hit rate without touching correctness.Why (evidence)
On the stacked id-comparison benchmark, the bloom brings dense id-insertion to HashMap parity at typical tag counts (0.99× at 7 tags, up from 0.91× without it), while dense still allocates ~half of HashMap. Cost: +8 B/op (the single
longfield). The demonstrating benchmark lands in the stacked id PR.Stacking
Dense-store id line: #11814 (Entry-less storage) → this (bloom fast-path) → id API + comparison benchmark. Based on
dougqh/dense-store.Follow-ups
Per-type graph coloring (raises hit rate); multi-level transitive prune for read-through (skip parents whose bit is clear).
🤖 Generated with Claude Code